Skip to content

Crowdstrike.fdr: map CommandHistory events to ECS process fields - #20285

Open
chemamartinez wants to merge 9 commits into
elastic:mainfrom
chemamartinez:crowdstrike-fdr-map-process-fields-commandhistory
Open

Crowdstrike.fdr: map CommandHistory events to ECS process fields#20285
chemamartinez wants to merge 9 commits into
elastic:mainfrom
chemamartinez:crowdstrike-fdr-map-process-fields-commandhistory

Conversation

@chemamartinez

@chemamartinez chemamartinez commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

Proposed commit message

crowdstrike.fdr: map CommandHistory events to ECS process fields

Parse CommandHistory events in the FDR ingest pipeline to populate
ECS process fields. The CommandHistory string (commands delimited by
the pilcrow character ¶) is mapped to process.command_line and split
into an array stored at crowdstrike.CommandHistory.

CrowdStrike uses U+00B6 PILCROW SIGN (¶) as the separator between 
individual commands in the CommandHistory field.
This character has no formal documentation from CrowdStrike,
but it is consistently observed in FDR CommandHistory events collected
in real scenarios.

The UserName field is copied to process.user.name for CommandHistory
events. A new script processor derives process.interactive from LogonType,
treating types 2 (Interactive), 10 (RemoteInteractive), 11 (CachedInteractive)
and 12 (CachedRemoteInteractive) as interactive sessions.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.
  • I have verified that any added dashboard complies with Kibana's Dashboard good practices

@chemamartinez chemamartinez self-assigned this Jul 22, 2026
@chemamartinez chemamartinez added enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Jul 22, 2026
@chemamartinez
chemamartinez marked this pull request as ready for review July 22, 2026 14:23
@chemamartinez
chemamartinez requested review from a team as code owners July 22, 2026 14:23
@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@github-actions

Copy link
Copy Markdown
Contributor

✅ Elastic Docs Style Checker (Vale)

No issues found on modified lines!


The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.

- script:
tag: script_process_interactive_logon_type_9b3d6c7a
description: Set process.interactive from crowdstrike.LogonType; types 2 (Interactive), 10 (RemoteInteractive), and 11 (CachedInteractive) indicate an interactive shell session.
if: ctx.crowdstrike?.LogonType instanceof String && ctx.crowdstrike.LogonType != ''

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:1148

The process.interactive script fires on any event carrying a LogonType, so authentication events (UserLogon/UserLogoff) get a bare process.interactive with no process context; gate it on an established process instead.

Details

The new script's condition is only ctx.crowdstrike?.LogonType instanceof String && ctx.crowdstrike.LogonType != '', which is not scoped to process events. LogonType is present on Windows authentication events, so this stamps process.interactive onto documents that have no process. This is observable in the regenerated fixtures: in test-fdr.log-expected.json a UserLogoff event (event.category: ["authentication"]) now gains a lone process: { "interactive": false } object, and test-user-map / test-windows show the same on non-process events. The effect is also broader than the changelog entry ("Map CommandHistory events to ECS process fields") describes. Attaching a process attribute to an authentication event is semantically misleading and creates a process object where none belongs.

Recommendation:

Gate the script on an already-established process so it only applies to genuine process events (CommandHistory and ProcessRollup set process.entity_id from TargetProcessId earlier in the pipeline; authentication events do not):

- script:
    tag: script_process_interactive_logon_type_9b3d6c7a
    description: Set process.interactive from crowdstrike.LogonType; types 2 (Interactive), 10 (RemoteInteractive), and 11 (CachedInteractive) indicate an interactive shell session.
    if: >-
      ctx.crowdstrike?.LogonType instanceof String && ctx.crowdstrike.LogonType != '' &&
      ctx.process?.entity_id != null
    source: |-
      def logonType = ctx.crowdstrike.LogonType;
      ctx.process.interactive = (logonType == '2' || logonType == '10' || logonType == '11');
    on_failure:
      - append:
          field: error.message
          value: "Failed to set process.interactive from LogonType: {{{_ingest.on_failure_message}}}"

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@elastic-vault-github-plugin-prod

elastic-vault-github-plugin-prod Bot commented Jul 22, 2026

Copy link
Copy Markdown
Contributor

🚀 Benchmarks report

Package crowdstrike 👍(6) 💚(7) 💔(1)

Expand to view
Data stream Previous EPS New EPS Diff (%) Result
falcon 5102.04 3378.38 -1723.66 (-33.78%) 💔

To see the full report comment with /test benchmark fullreport

@andrewkroh andrewkroh added the documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. label Jul 22, 2026
Comment on lines +986 to +988
if (ctx.process == null) {
ctx.process = new HashMap();
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
if (ctx.process == null) {
ctx.process = new HashMap();
}
ctx.process = ctx.process ?: [:];

source: |-
def logonType = ctx.crowdstrike.LogonType;
if (ctx.process == null) ctx.process = new HashMap();
ctx.process.interactive = (logonType == '2' || logonType == '10' || logonType == '11');

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this wants a comment explaining the logon types.

@chemamartinez
chemamartinez requested a review from efd6 July 23, 2026 06:34
Comment thread packages/crowdstrike/data_stream/fdr/fields/fields.yml Outdated
// 7=Unlock, 8=NetworkCleartext, 9=NewCredentials, 10=RemoteInteractive,
// 11=CachedInteractive, 12=CachedRemoteInteractive, 13=CachedUnlock
def logonType = ctx.crowdstrike.LogonType;
ctx.process.interactive = (logonType == '2' || logonType == '10' || logonType == '11' || logonType == '12');

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟠 High confidence: medium path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:3258

process.interactive is derived from crowdstrike.LogonType on every event that has any process object, which labels processes that are not the session's shell — restrict the processor to CommandHistory events.

Details

LogonType is an attribute of the Windows logon session, but ECS defines process.interactive per-process ("inferred from the processes file descriptors... the character device for the controlling tty is the same as stdin and stderr") — the definition this PR adds to docs/README.md line 3197. Applying the session's LogonType to whatever process object happens to be on the event produces wrong values, and the PR's own fixtures show it:

  • packages/crowdstrike/data_stream/fdr/_dev/test/pipeline/test-windows.log-expected.json line 11952 now asserts "interactive": true for the Windows System process (event_simpleName LogonBehaviorCompositionDetectInfo, ImageFileName/CommandLine "System", entity_id 21475403566). System is the kernel process; it has no controlling tty and can never be attached to an interactive shell. It is marked interactive purely because the session's LogonType is 2.
  • test-fdr.log-expected.json lines 8069, 8557, 9370 and test-windows.log-expected.json lines 8756, 8963, 9153 add "interactive": false to UserLogon/authentication events whose process object is only a ContextProcessId reference (entity_id plus thread.id). ECS false means "known to be non-interactive"; here interactivity is simply unknown, so the pipeline is asserting a fact the source event does not carry.

The ctx.process != null gate added in this revision keeps the field off events with no process object at all, but it does not distinguish the process the logon session actually belongs to from an unrelated context/target process reference, so both cases above still fire.

Recommendation:

Scope the processor to the event type this PR is about (CommandHistory), where the recorded shell genuinely is the process the logon session belongs to:

  - script:
      tag: script_process_interactive_logon_type_9b3d6c7a
      description: >-
        Set process.interactive from crowdstrike.LogonType for CommandHistory
        events only. On other FDR events LogonType describes the logon session,
        not the process referenced by ContextProcessId/TargetProcessId, so the
        value would not describe that process. Logon types 2 (Interactive),
        10 (RemoteInteractive), 11 (CachedInteractive) and 12
        (CachedRemoteInteractive) indicate an interactive shell session.
      if: >-
        ctx.crowdstrike?.event_simpleName == 'CommandHistory' &&
        ctx.crowdstrike?.LogonType instanceof String &&
        ctx.crowdstrike.LogonType != '' &&
        ctx.process != null
      source: |-
        def logonType = ctx.crowdstrike.LogonType;
        ctx.process.interactive = (logonType == '2' || logonType == '10' || logonType == '11' || logonType == '12');
      on_failure:
        - append:
            field: error.message
            value: "Failed to set process.interactive from LogonType: {{{_ingest.on_failure_message}}}"

This reverts the added "interactive" lines in test-fdr.log-expected.json and test-windows.log-expected.json and keeps only the two CommandHistory assertions in test-fdr-command-history.log-expected.json.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That's intentional. We want to map process.interactive in every event it contains crowdstrike.LogonType and contains a process object.

if (!remaining.trim().isEmpty()) {
commands.add(remaining.trim());
}
ctx.process.command_line = commands.size() > 0 ? commands.get(0) : history.trim();

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:1083

process.command_line is set to commands.get(0), an arbitrary entry of a multi-command history buffer, silently dropping the rest — only populate it when the history holds a single command.

Details

For the two-command fixture in test-fdr-command-history.log, process.command_line becomes "sc stop CSFalconService" and the second command, "Stop-Service -Name CSFalconService -Force", never reaches process.command_line (test-fdr-command-history.log-expected.json line 102). Detection rules and searches on process.command_line therefore see only one of the N commands, chosen by position.

Index 0 is not a defensible choice: the same event carries FirstCommand, LastAdded, LastDisplayed, CommandSequence and CommandCount precisely because the history is an indexed rolling buffer, so element 0 is not reliably the newest, the oldest, or the command of interest. The previous revision assigned the whole delimited string; this revision narrowed it to element 0, which trades one wrong value for another.

ECS defines process.command_line as "Full command line that started the process". The process on these events is identified by TargetProcessId (process.entity_id 7292754410393 in the fixture) — the shell — and its own command line is none of the history entries. A related symptom: this script is placed at line 1054, after split_command_line_c3beef26 (line 972), so unlike every other source of process.command_line in this pipeline the value produces no process.args/process.args_count; the fixture's process object has command_line but no args.

The full parsed history is already preserved at crowdstrike.CommandHistory, so nothing is lost by not guessing.

Recommendation:

Keep the parsed array and only expose process.command_line when it is unambiguous:

      source: |-
        def history = ctx.crowdstrike.CommandHistory;
        def commands = new ArrayList();
        def remaining = history;
        int idx = remaining.indexOf("\u00b6");
        while (idx >= 0) {
          def cmd = remaining.substring(0, idx).trim();
          if (!cmd.isEmpty()) {
            commands.add(cmd);
          }
          remaining = remaining.substring(idx + 1);
          idx = remaining.indexOf("\u00b6");
        }
        if (!remaining.trim().isEmpty()) {
          commands.add(remaining.trim());
        }
        ctx.crowdstrike.CommandHistory = commands;
        // A history with several entries has no single command line; picking an
        // element by position would drop the others from process.command_line.
        if (commands.size() == 1) {
          ctx.process = ctx.process ?: [:];
          ctx.process.command_line = commands.get(0);
        }

If you do keep populating process.command_line, move this script above rename_crowdstrike_CommandLine_to_process_command_line_307047e3 (line 967) so split_command_line_c3beef26 also derives process.args and process.args_count, as it does for every other command line in this pipeline.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

- name: text
type: match_only_text
- name: CommandSequence
type: match_only_text

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🔵 Low confidence: medium path: packages/crowdstrike/data_stream/fdr/fields/fields.yml:233

crowdstrike.CommandSequence is declared as match_only_text, so the sequence index cannot be filtered exactly, sorted or aggregated — declare it as keyword.

Details

CommandSequence carries the position of the entry within the CrowdStrike command-history buffer ("1" in both events of test-fdr-command-history.log). match_only_text is a text type: it is not doc_values-backed, so the field cannot be used in a terms aggregation, cannot be sorted on, and cannot be matched with an exact term query — all of which are the natural uses of a sequence index when reconstructing command order.

The adjacent CommandCount and CommandCountMax entries (fields.yml lines 221-226) are also match_only_text, but each sits above a commented-out type: long declaration, which marks them as a known legacy compromise rather than the convention to follow for a newly added field.

Recommendation:

Declare the sequence index as a keyword:

    - name: CommandSequence
      type: keyword

The docs/README.md row for crowdstrike.CommandSequence needs regenerating with elastic-package build after the change.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@elastic-vault-github-plugin-prod

Copy link
Copy Markdown
Contributor

✅ All changelog entries have the correct PR link.

}
ctx.crowdstrike.CommandHistory = commands;
ctx.process = ctx.process ?: [:];
ctx.process.command_line = commands;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🟡 Medium confidence: medium path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:1083

process.command_line is now assigned the whole command array, so the field is multi-valued on CommandHistory events but a scalar string everywhere else in the data stream; populate it only when the history holds exactly one command.

Details

ECS defines process.command_line as the full command line that started the process identified by process.entity_id. On a CommandHistory event process.entity_id is TargetProcessId (the shell), and crowdstrike.CommandHistory is the buffer of commands typed inside that shell, not the command line that launched it. Assigning the array puts N unrelated command strings into a single-valued ECS field.

This is visible in the committed fixtures: test-fdr-command-history.log-expected.json now carries process.command_line as a two-element array, and test-windows.log-expected.json turns a previously scalar value into a one-element array, while every other event in the same expected file (for example the LogonBehaviorCompositionDetectInfo document with "command_line": "System") keeps a scalar string. The field is therefore scalar on most documents and multi-valued on CommandHistory documents. In ES|QL most functions and aggregations return null and emit a warning on multi-valued rows, so queries and rules over process.command_line behave differently for these documents. The events also get process.command_line without the matching process.args / process.args_count that split_command_line produces for every other command line.

This also moves away from the direction already agreed on this PR (populate process.command_line only when the history holds a single command); the previous revision's data-loss problem is better solved by leaving the field unset for multi-command buffers than by making it an array.

Recommendation:

Keep the full list in crowdstrike.CommandHistory and only promote it to process.command_line when it is unambiguous:

        if (!remaining.trim().isEmpty()) {
          commands.add(remaining.trim());
        }
        ctx.crowdstrike.CommandHistory = commands;
        if (commands.size() == 1) {
          ctx.process = ctx.process ?: [:];
          ctx.process.command_line = commands.get(0);
        }

Then regenerate test-fdr-command-history.log-expected.json and test-windows.log-expected.json so the two-command document has no process.command_line and the single-command documents carry a scalar string.


🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

description: >-
Split crowdstrike.CommandHistory on U+00B6 PILCROW SIGN into an array
and assign the result to both crowdstrike.CommandHistory and
process.command_line. Placed after split_command_line so that processor

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🔵 Low confidence: high path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:1058

The rewritten description misstates why the processor is safe and drops the note that the pilcrow delimiter is undocumented by CrowdStrike; restore the delimiter rationale and describe the actual guard.

Details

The description now claims the processor is "Placed after split_command_line so that processor only runs on string-valued command lines from crowdstrike.CommandLine." That is not what the ordering does: split_command_line reads process.command_line (populated from crowdstrike.CommandLine), and it is the ctx.crowdstrike?.CommandHistory instanceof String condition, not the position, that keeps this processor off already-converted array values. The sentence is also grammatically broken.

The previous revision recorded that U+00B6 PILCROW SIGN is not formally documented by CrowdStrike but is consistently observed as the delimiter in FDR CommandHistory events. That is the non-obvious reason a magic character appears in the source, and it is the one piece of context a future maintainer cannot recover from the code; the rewrite removed it.

Recommendation:

Restore the delimiter rationale and state the real guard:

  - script:
      description: >-
        Split crowdstrike.CommandHistory into an array of individual commands
        and mirror it to process.command_line. CrowdStrike uses U+00B6 PILCROW
        SIGN as the delimiter between commands in the history string; this
        character has no formal documentation from CrowdStrike but is
        consistently observed in FDR CommandHistory events. The
        `instanceof String` guard keeps the processor idempotent once the
        field has been converted to an array.
      tag: script_command_history_to_process_command_line_8c3e1f92

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

ctx.process = ctx.process ?: [:];
ctx.process.command_line = commands;
on_failure:
- append:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Severity: 🔵 Low confidence: high path: packages/crowdstrike/data_stream/fdr/elasticsearch/ingest_pipeline/default.yml:1085

The on_failure append processors added by this PR have no tag and use an ad-hoc error message instead of the standard ingest.on_failure* template every other handler in this pipeline uses; align them with the existing convention.

Details

Both new script processors (script_command_history_to_process_command_line_8c3e1f92 here, and script_process_interactive_logon_type_9b3d6c7a further down) attach an on_failure append that omits the tag field and writes a custom string carrying only _ingest.on_failure_message.

Every other on_failure handler in this pipeline uses a tagged append with the full template that also records the failing processor type, tag and pipeline, for example append_error_message_4ef54c75 on the json processor. Without those fields an error.message from these handlers cannot be traced back to the processor that produced it, and the untagged processors will be flagged by elastic-package check once the package moves to format_version 3.6.0 or later.

Recommendation:

Use the pipeline's standard tagged handler on both new scripts:

      on_failure:
        - append:
            tag: append_error_message_8c3e1f92
            field: error.message
            value: 'Processor {{{_ingest.on_failure_processor_type}}} with tag {{{_ingest.on_failure_processor_tag}}} in pipeline {{{_ingest.on_failure_pipeline}}} failed with message: {{{_ingest.on_failure_message}}}'

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@vera-review-bot

Copy link
Copy Markdown

Review summary

Issues found across the latest commits d5f78fef5cd816 (12 commits) — 1 medium, 2 low
  • 🟡 process.command_line is now assigned the whole command array, so the field is multi-valued on CommandHistory events but a scalar string everywhere else in the data stream (link) (Unresolved)
  • 🔵 The rewritten description misstates why the processor is safe and drops the note that the pilcrow delimiter is undocumented by CrowdStrike (link) (Unresolved)
  • 🔵 The on_failure append processors added by this PR have no tag and use an ad-hoc error message instead of the standard ingest.on_failure* template every other handler in this pipeline uses (link) (Unresolved)
Issues found across earlier commits c9f2ec754efc99 (34 commits) — 1 high, 1 medium, 1 low
  • 🟠 process.interactive is derived from crowdstrike.LogonType on every event that has any process object, which labels processes that are not the session's shell — restrict the processor to CommandHistory events. (link) (Unresolved)
  • 🟡 process.command_line is set to commands.get(0), an arbitrary entry of a multi-command history buffer, silently dropping the rest — only populate it when the history holds a single command. (link) (Unresolved)
  • 🔵 crowdstrike.CommandSequence is declared as match_only_text, so the sequence index cannot be filtered exactly, sorted or aggregated — declare it as keyword. (link) (Unresolved)
Issues found across earlier commits d0257c7 — 1 medium
  • 🟡 The process.interactive script fires on any event carrying a LogonType, so authentication events (UserLogon/UserLogoff) get a bare process.interactive with no process context (link) (Unresolved)

A new commit triggers another review — at most once every 15 minutes. I skip the PR while it's approved or has merge conflicts.

🤖 AI-Generated Review | Vera Review Bot | 📚 Knowledge base: integration-skills

⚠️ Automated review — verify suggestions before applying.

@infra-vault-gh-plugin-prod

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @chemamartinez

@navnit-elastic navnit-elastic left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, Thanks! Please wait for Dan.

@mergify

mergify Bot commented Jul 29, 2026

Copy link
Copy Markdown
Contributor

Tick the box to add this pull request to the merge queue (same as @mergifyio queue).

  • Queue this pull request

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation. Applied to PRs that modify *.md files. enhancement New feature or request Integration:crowdstrike CrowdStrike Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants